Illumination control for vehicle sensors

ABSTRACT

A system includes an illumination source, a camera arranged to detect illumination from the illumination source, and a computer communicatively coupled to the illumination source and the camera. The computer is programmed to detect an object in image data from the camera, identify landmarks of the object in the image data, determine a distance from the camera to the object based on a pixel distance between the landmarks in the image data, and adjust a brightness of the illumination source based on the distance.

BACKGROUND

Vehicles are often equipped with cameras. The cameras detectelectromagnetic radiation in some range of wavelengths. The wavelengthscan be visible light, infrared radiation, ultraviolet light, or somerange of wavelengths including visible, infrared, and/or ultravioletlight. The cameras include image sensors such as charge-coupled devices(CCD), active-pixel sensors such as complementary metal-oxidesemiconductor (CMOS) sensors, etc.

For situations in which the ambient environment is insufficientlyilluminated for the cameras, vehicles are equipped with illuminationsources. The illumination sources are arranged to illuminate areas inthe fields of view of the cameras. The cameras thereby receiveillumination from the illumination sources reflected from features ofthe environment.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an example vehicle.

FIG. 2 is a top view of the example vehicle with a passenger cabinexposed for illustration.

FIG. 3 is an example frame of image data from a camera of the vehicle.

FIG. 4 is a process flow diagram of an example process for controllingillumination sources of the vehicle.

DETAILED DESCRIPTION

The system described herein provides good illumination of an environmentfor a camera while keeping the illumination within appropriate levelsfor a person in the environment. The illumination can come from anillumination source with an adjustable brightness. Greater illuminationof an environment by the illumination source permits more information tobe discerned from image data of the environment generated by the camera.At the same time, the illumination source should be kept at anappropriate level for persons in the environment. This can beparticularly important for illumination outside the visible spectrumbecause in this range a person's defensive physiological responses suchas narrowing their pupils may not occur. The system described hereindetermines a distance to the person and adjusts the brightness of theillumination source based on the distance. The brightness can thus bemaximized for the distance of the person, rather than keeping thebrightness at a constant, low level that is appropriate even at shortdistances (e.g., because of the aforementioned concerns about a person'sability to react or adjust to light outside the visible spectrum).Moreover, the brightness can be adjusted in real-time, e.g., loweringthe brightness as the person moves closer to the camera. Furthermore,the distance is determined based on image data from the camera of theperson, meaning that no components are needed besides the camera and theillumination source. The cost and complexity of the system is thus keptlow. The system is described below with respect to an automotivecontext. The system is also useful in other contexts such as securityand monitoring systems, doorbell camera systems, surveillance systems,etc.

A system includes an illumination source, a camera arranged to detectillumination from the illumination source, and a computercommunicatively coupled to the illumination source and the camera. Thecomputer is programmed to detect an object in image data from thecamera, identify landmarks of the object in the image data, determine adistance from the camera to the object based on a pixel distance betweenthe landmarks in the image data, and adjust a brightness of theillumination source based on the distance.

The illumination source may be configured to produce illuminationoutside a visible range. The illumination source may be configured toproduce infrared illumination.

A computer includes a processor and a memory storing instructionsexecutable by the processor to detect an object in image data from acamera, identify landmarks of the object in the image data, determine adistance from the camera to the object based on a pixel distance betweenthe landmarks in the image data, and adjust a brightness of anillumination source based on the distance. The illumination source isarranged to produce illumination detectable by the camera.

The object may be a human face. The instructions may further includeinstructions to, after adjusting the brightness of the illuminationsource and receiving new image data, perform facial recognition on thenew image data of the face. The instructions may further includeinstructions to actuate a vehicle component upon the facial recognitionrecognizing the face as a recognized face.

The instructions may further include instructions to determine whether afeature of the face is three-dimensional. The illumination source may bea first illumination source, and the instructions may further includeinstructions to illuminate a second illumination source while receivingthe new image data.

Adjusting the brightness of the illumination source may includeadjusting the brightness to a brightness level, and the brightness levelmay have an increasing relationship with the distance.

Determining the distance from the camera to the object may includemultiplying the pixel distance by a prestored distance-to-pixel-distanceratio. The prestored distance-to-pixel-distance ratio may be based on atleast a 90th percentile size of a statistical distribution of objects ofa same type as the object.

The instructions may further include instructions to determine whetherthe object is sufficiently illuminated for identifying the landmarks ofthe object. The instructions may further include instructions toincrease the brightness of the illumination source upon determining thatthe object is insufficiently illuminated for identifying the landmarksof the object. The instructions may further include instructions todecrease the brightness of the illumination source within a prestoredduration after increasing the brightness. The prestored duration may beat most a duration in which the camera can capture five frames.

Increasing the brightness may include increasing the brightness to aprestored brightness level, and the instructions may further includeinstructions to prevent the brightness of the illumination source frombeing set to the prestored brightness level for longer than theprestored duration.

Increasing the brightness may include increasing the brightness at aprestored rate while receiving additional image data. The instructionsmay further include instructions to maintain the brightness at a currentbrightness level upon determining that the object is sufficientlyilluminated for identifying the landmarks of the object.

A method includes detecting an object in image data from a camera,identifying landmarks of the object in the image data, determining adistance from the camera to the object based on a pixel distance betweenthe landmarks in the image data, and adjusting a brightness of anillumination source based on the distance. The illumination source isarranged to produce illumination detectable by the camera.

With reference to the Figures, a system 102 of a vehicle 100 includes atleast one illumination source 104, at least one camera 106 arranged todetect illumination from one or more of the at least one illuminationsource 104, and a computer 108 communicatively coupled to the at leastone illumination source 104 and the at least one camera 106. Thecomputer 108 is programmed to detect an object 110 in image data 112from the at least one camera 106, identify landmarks 114 of the object110 in the image data 112, determine a distance D from the at least onecamera 106 to the object 110 based on a pixel distance d between thelandmarks 114 in the image data 112, and adjust a brightness of the atleast one illumination source 104 based on the distance D.

With reference to FIG. 1, the vehicle 100 may be any passenger orcommercial automobile such as a car, a truck, a sport utility vehicle, acrossover, a van, a minivan, a taxi, a bus, etc.

The vehicle 100 may be an autonomous vehicle. A vehicle computer can beprogrammed to operate the vehicle 100 independently of the interventionof a human operator, completely or to a lesser degree. The vehiclecomputer may be programmed to operate the propulsion, brake system,steering system, and/or other vehicle systems based on data from the atleast one camera 106 and other sensors. For the purposes of thisdisclosure, autonomous operation means the vehicle computer controls thepropulsion, brake system, and steering system without input from a humanoperator; semi-autonomous operation means the vehicle computer controlsone or two of the propulsion, brake system, and steering system and ahuman operator controls the remainder; and nonautonomous operation meansa human operator controls the propulsion, brake system, and steeringsystem.

The computer 108 is a microprocessor-based computing device, e.g., ageneric computing device including a processor and a memory, anelectronic controller or the like, a field-programmable gate array(FPGA), an application-specific integrated circuit (ASIC), etc. Thecomputer 108 can thus include a processor, a memory, etc. The memory ofthe computer 108 can include media for storing instructions executableby the processor as well as for electronically storing data and/ordatabases, and/or the computer 108 can include structures such as theforegoing by which programming is provided. The computer 108 can bemultiple computers coupled together. The computer 108 may be the same asthe vehicle computer or may be separate from the vehicle computer.

The computer 108 may transmit and receive data through a communicationsnetwork 116 such as a controller area network (CAN) bus, Ethernet, WiFi,Local Interconnect Network (LIN), onboard diagnostics connector(OBD-II), and/or by any other wired or wireless communications network.The computer 108 may be communicatively coupled to the cameras 106, theillumination sources 104, and other vehicle components 118 via thecommunications network 116.

The cameras 106 detect electromagnetic radiation in some range ofwavelengths. For example, the cameras 106 may detect visible light,infrared radiation, ultraviolet light, or some range of wavelengthsincluding visible, infrared, and/or ultraviolet light. For example, thecameras 106 can include image sensors such as charge-coupled devices(CCD), active-pixel sensors such as complementary metal-oxidesemiconductor (CMOS) sensors, etc. The cameras 106 are configured todetect illumination from respective illumination sources 104; i.e., therange of wavelengths of electromagnetic radiation detectable by thecamera 106 completely or significantly overlap the range of wavelengthsproduced by the respective illumination source 104.

The illumination sources 104 can produce illumination in some range ofwavelengths, specifically, illumination detectable by the cameras 106.For example, the illumination sources 104 may produce visible light,infrared radiation, ultraviolet light, or some range of wavelengthsincluding visible, infrared, and/or ultraviolet light. The illuminationsources 104 are configured to produce illumination in a range ofwavelengths completely or significantly encompassed by the range ofwavelengths detectable by the cameras 106. For example, the illuminationsources 104 can produce and the cameras 106 can detect illuminationoutside a visible range, e.g., infrared illumination, e.g.,near-infrared illumination (700-1300 nanometers (nm)). The illuminationsources 104 can be any suitable types for producing the desiredwavelengths, e.g., for visible light, tungsten, halogen, high-intensitydischarge (HID) such as xenon, light-emitting diodes (LED), etc.; forinfrared light, LEDs, lasers, filtered incandescent, etc.

The vehicle 100 includes the vehicle components 118 that are actuatableby the computer 108 in response to image data 112 from the cameras 106,as described below. Examples of vehicle components 118 include doorlocks 120, seats 122, a climate-control system 124, etc., as aredescribed in detail below. Other vehicle components 118 besides theseexamples can be actuatable by the computer 108 in response to image data112 from the cameras 106.

With reference to FIG. 2, the vehicle 100 includes a body 128. Thevehicle 100 may be of a unibody construction, in which a frame and thebody 128 of the vehicle 100 are a single component. The vehicle 100 may,alternatively, be of a body-on-frame construction, in which the framesupports the body 128 that is a separate component from the frame. Theframe and the body 128 may be formed of any suitable material, forexample, steel, aluminum, etc.

The door locks 120 are engageable to permit or prevent doors 126 of thevehicle 100 from being opened. The door locks 120 are movable between adisengaged position, in which doors 126 are unlocked, i.e., permitted toopen if the door handle is operated, and an engaged position, in whichthe doors 126 are locked, i.e., prevented from opening even if the doorhandle is operated.

The vehicle 100 includes a passenger cabin 130 to house occupants, ifany, of the vehicle 100. The passenger cabin 130 includes one or more ofthe seats 122 disposed in a front row of the passenger cabin 130 and oneor more of the seats 122 disposed in a second row behind the front row.The passenger cabin 130 may also include third-row seats 122 (not shown)at a rear of the passenger cabin 130. In FIG. 2, the front-row seats 122are shown to be bucket seats, but the seats 122 may be other types. Theposition and orientation of the seats 122 and components thereof may beadjustable by an occupant.

Each seat 122 can include actuators for adjusting the seat 122 inmultiple degrees of freedom, e.g., a tilt of the seat 122, a height ofthe seat 122, a recline angle of the seat 122, or a lumbar supportposition of the seat 122. The tilt of the seat 122 is an angle of a seatbottom 132 of the seat 122 relative to the passenger cabin 130 about alateral axis, i.e., a pitch of the seat bottom 132. The height of theseat 122 is a vertical distance of a reference point on the seat bottom132 relative to the passenger cabin 130. The recline angle of the seat122 is an angle of a seat back 134 of the seat 122 relative to the seatbottom 132. The lumbar support position is a vehicle-forward position ofa lumbar support bar (not shown), located in the seat back 134, relativeto the seat back 134. Additionally or alternatively, the seat 122 may beadjustable in other degrees of freedom.

The climate-control system 124 provides heating and/or cooling to thepassenger cabin 130 of the vehicle 100. The climate-control system 124may include a compressor, a condenser, a receiver-dryer, athermal-expansion valve, an evaporator, blowers, fans, ducts, vents,vanes, temperature sensors, and other components that are known forheating or cooling vehicle interiors. The climate-control system 124 mayoperate to cool the passenger cabin 130 by transporting a refrigerantthrough a heat cycle to absorb heat from the passenger cabin 130 andexpel the heat from the vehicle 100, as is known. The climate-controlsystem 124 may include a heater core that operates as a radiator for anengine of the vehicle 100 by transferring some waste heat from theengine into the passenger cabin 130, as is known. The climate-controlsystem 124 may include an electrically powered heater such as aresistive heater, positive-temperature-coefficient heater, electricallypowered heat pump, etc.

The cameras 106 are typically arranged in positions to detect persons inthe vicinity of the vehicle 100, e.g., occupants and/or pedestrians. Forexample, the cameras 106 can include a first camera 106 a with a fieldof view encompassing an area in front of the vehicle 100. The firstcamera 106 a can be mounted on or above a rear-view mirror and aimed ina vehicle-forward direction. For another example, the cameras 106 caninclude a second camera 106 b with a field of view encompassing anoperator of the vehicle 100. The second camera 106 b can be mounted toan instrument panel and aimed in a vehicle-rearward direction, as shownin FIG. 3. The second camera 106 b can be arranged to include one ormore occupants, e.g., only the operator, or all the occupants, etc., ofthe passenger cabin 130. For another example, the cameras 106 caninclude a third camera 106 c with a field of view encompassing an areanext to the doors 126 of the vehicle 100. The third camera 106 c can bemounted to a B-pillar of the vehicle 100 and aimed in a vehicle-lateraldirection. A person approaching the door 126 will be in the field ofview of the third camera 106 c.

The illumination sources 104 are arranged to produce illuminationdetectable by the cameras 106, and likewise the cameras 106 are arrangedto detect illumination from the illumination sources 104. Specifically,the illumination sources 104 are arranged to illuminate areas in thefields of view of the cameras 106, and the cameras 106 are arranged sothat the fields of view of the cameras 106 encompass areas illuminatedby the illumination sources 104. The cameras 106 thereby receiveillumination from the illumination sources 104 that has reflected off offeatures of the environment. For example, the illumination sources 104can each be mounted to a respective one of the cameras 106 and aimed inthe same direction as that camera 106. The respective pairings ofillumination sources 104 and cameras 106 can be packaged as a singleunit. The illumination sources 104 can include a first illuminationsource 104 a mounted to the first camera 106 a, a second illuminationsource 104 b mounted to the second camera 106 b, a third illuminationsource 104 c mounted to the third camera 106 c, and so on.

With reference to FIG. 3, the cameras 106 generate image data 112 of therespective fields of view of the cameras 106. The image data 112 are asequence of image frames of the fields of view of the respective cameras106. FIG. 3 shows an example image frame of a person's face. Each imageframe is a two-dimensional matrix of pixels. Each pixel has a brightnessor color represented as one or more numerical values, e.g., a scalarunitless value of photometric light intensity between 0 (black) and 1(white), or values for each of red, green, and blue, e.g., each on an8-bit scale (0 to 255) or a 12- or 16-bit scale. The pixels may be a mixof representations, e.g., a repeating pattern of scalar values ofintensity for three pixels and a fourth pixel with three numerical colorvalues, or some other pattern. For example, FIG. 3 is an image frame inwhich each pixel is a scalar value of intensity of illumination innear-infrared wavelengths. Position in an image frame, i.e., position inthe field of view of the camera 106 at the time that the image frame wasrecorded, can be specified in pixel dimensions or coordinates, e.g., anordered pair of pixel distances, such as a number of pixels from a topedge and a number of pixels from a left edge of the field of view.

The image data 112 can be of objects 110 that are in the field of viewof one of the cameras 106. One such object 110 is a human face, as shownin FIG. 3. The objects 110 can include landmarks 114. For the purposesof this disclosure, a “landmark” is defined as a predefined feature oneach object 110 of a specific type, which is shared by objects 110 ofthat type. For example, if the type of object 110 is a human face,possible landmarks 114 are a center of the nose, a bottom of the ear, acorner of the mouth, etc.

FIG. 4 is a process flow diagram illustrating an exemplary process 400for controlling the illumination sources 104. The memory of the computer108 stores executable instructions for performing the steps of theprocess 400 and/or programming can be implemented in structures such asmentioned above. The process 400 can be performed individually for eachof the cameras 106. As a general overview of the process 400, thecomputer 108 receives image data 112 from the camera 106 and determineswhether an object 110, such as a human face, is present in the imagedata 112. If so, the computer 108 determines whether there is sufficientillumination for a distance determination and increases the illuminationfrom the illumination source 104 paired with the camera 106 if not. Forthe distance determination, the computer 108 identifies pixel locationsof landmarks 114 of the object 110, determines a pixel distance dbetween the pixel locations, and determines the distance D to the object110 using the pixel distance d. The computer 108 adjusts the brightnessof the illumination source 104 paired with the camera 106 based on thedistance D. The computer 108 activates multiple illumination sources 104while receiving image data 112 from the camera 106. Using the new imagedata 112, the computer 108 determines whether the object 110 isrecognized, e.g., as an authorized user, and whether the object 110 isthree-dimensional. If so, the computer 108 actuates one of the vehiclecomponents 118. The process 400 is performed continuously while thevehicle 100 is on.

The process 400 begins in a block 405, in which the computer 108receives image data 112 from the camera 106, e.g., receives an imageframe from the camera 106.

Next, in a decision block 410, the computer 108 detects whether anobject 110, e.g., a human face, is in the image data 112, e.g., usingfacial detection. The computer 108 can detect the human face in theimage data 112 by using any suitable facial-detection technique, e.g.,knowledge-based techniques such as a multiresolution rule-based method;feature-invariant techniques such as grouping of edges, space gray-leveldependence matrix, or mixture of Gaussian; template-matching techniquessuch as shape template or active shape model; or appearance-basedtechniques such as eigenface decomposition and clustering, Gaussiandistribution and multilayer perceptron, neural network, support vectormachine with polynomial kernel, a naive Bayes classifier with jointstatistics of local appearance and position, higher order statisticswith hidden Markov model, or Kullback relative information. If no object110 is detected, the process 400 returns to the block 405 to continuemonitoring the image data 112. If an object 110 is detected, the process400 proceeds to a decision block 415.

In the decision block 415, the computer 108 determines whether theobject 110 is sufficiently illuminated for identifying the landmarks 114of the object 110. The object 110 can be illuminated both by theillumination sources 104 and by ambient sources of illumination. Forexample, the computer 108 can average the light intensity of each pixelof the image frame, or of each pixel that is part of the object 110detected in the decision block 410. The average brightness can becompared with a brightness threshold, and the object 110 is sufficientlyilluminated if the brightness is above the brightness threshold andinsufficiently illuminated if the brightness is below the brightnessthreshold. The brightness threshold can be chosen as a minimumbrightness for the computer 108 to successfully identify the landmarks114 in a block 430 below. For another example, the computer 108 canperform the identification of the landmarks 114 described below withrespect to the block 430. If the computer 108 successfully identifiesthe landmarks 114, then the object 110 is sufficiently illuminated, andif not, then the object 110 is insufficiently illuminated. If the object110 is insufficiently illuminated, the process 400 proceeds to adecision block 420. If the object 110 is sufficiently illuminated, theprocess 400 proceeds to the block 430.

In the decision block 420, the computer 108 determines whether one ofthe illumination sources 104 has reached a maximum level ofillumination. Specifically, the illumination source 104 can be theillumination source 104 that is paired with the camera 106, e.g., thefirst illumination source 104 a for the first camera 106 a, the secondillumination source 104 b for the second camera 106 b, the thirdillumination source 104 c for the third camera 106 c, etc. The maximumillumination is a brightness level of the illumination source 104 chosento be an appropriate, e.g., to avoid damaging a person's eyes,brightness level for a person near the illumination source 104 for theinterval of time for which the illumination source 104 will remainilluminated at that brightness level. For example, the maximum level ofillumination can be brighter for very short intervals. If theillumination source 104 has already reached the maximum level ofillumination, the process 400 ends. If the illumination source 104 hasnot yet reached the maximum level of illumination, the process 400proceeds to a block 425.

In the block 425, the computer 108 increases the brightness of theillumination source 104. For example, the computer 108 can increase thebrightness at a prestored rate while receiving additional image data 112from the camera 106. The prestored rate is a change in brightness perunit time and can be chosen based on a rate at which the camera 106generates image frames of image data 112 and a rate at which thecomputer 108 determines whether the object 110 is sufficientlyilluminated in the decision block 415. The process 400 thus iteratesthrough the decision block 415, the decision block 420, and the block425, with a slight increase in brightness with each iteration, untileither the object 110 is sufficiently illuminated or the illuminationsource 104 reaches the maximum level of illumination. Upon determiningthat the object 110 is sufficiently illuminated for identifying thelandmarks 114 of the object 110 in the decision block 415, the computer108 maintains the brightness at a current brightness level. The computer108 can thus find the lowest brightness level at which the illuminationsource 104 sufficiently illuminates the object 110.

Remaining with the block 425, for another example, the computer 108 canincrease the brightness of illumination source 104 to a prestoredbrightness level and then decrease the brightness of the illuminationsource 104 within a prestored duration after increasing the brightness.The prestored brightness level can be the maximum level of illuminationfrom the decision block 420. The prestored duration can be a durationduring which the camera 106 can capture five frames at most, e.g., aduration in which the camera 106 can capture one or two frames. Thecomputer 108 prevents the brightness of the illumination source 104 frombeing set to the prestored brightness level for longer than theprestored duration. The computer 108 can thus use a very bright level ofillumination to provide sufficient illumination for the object 110 whilekeeping the duration of that level of illumination short enough to beappropriate for a person. After the block 425, the process 400 returnsto the decision block 415.

In the block 430, the computer 108 identifies the landmarks 114 of theobject 110 in the image data 112. For example, if the object 110 is ahuman face, the locations of the landmarks 114 can be outputted as aresult of the facial-detection technique used in the decision block 410above. The computer 108 can use the output from the decision block 410,or the computer 108 can run the facial-detection technique again, e.g.,if the brightness of the illumination source 104 was increased in theblock 425. As shown in FIG. 3, for example, the landmarks 114 are thenose and the ear of a human face. The locations of the landmarks 114within the image frame are specified in pixel coordinates, e.g., (p_(x),p_(y)), in which p_(x) is a horizontal pixel distance in the image frameand p_(y) is a vertical pixel distance in the image frame.

Next, in a block 435, the computer 108 determines the pixel distance dbetween the landmarks 114 in the image data 112. The pixel distance dcan be a Euclidean distance in pixel coordinates, e.g.,

$d = \sqrt{\left( {p_{x1} - p_{x2}} \right)^{2} - \left( {p_{y1} - p_{y2}} \right)^{2}}$in which (p_(x1), p_(y1)) are the pixel coordinates of one of thelandmarks 114 and (p_(x2), p_(y2)) are the pixel coordinates of theother of the landmarks 114.

Next, in a block 440, the computer 108 determines a distance D from thecamera 106 to the object 110 based on the pixel distance d between thelandmarks 114 in the image data 112. The computer 108 stores apredefined relationship between the distance D and the pixel distance d.For example, the predefined relationship can be linear, i.e., thedistance D is equal to the pixel distance d multiplied by a prestoreddistance-to-pixel-distance ratio R, i.e., D=Rd. The ratio R can be avalue stored in memory, i.e., a constant. The value of the ratio R canbe based on a known geometrical relationship between the pixel distanced, the physical distance between the features of the object 110corresponding to the landmarks 114 in the image data 112, and thedistance D to the object 110. The physical distance between the featuresvaries within a population of objects 110 of the same type as the object110 according to a statistical distribution, e.g., a normaldistribution. The value of the ratio R can be based on the physicaldistance between the features being at least a 90th percentile size,e.g., being a 95th percentile size, of the statistical distribution ofthe objects 110 of the same type as the object 110; in other words, the,e.g., 95th percentile size of the physical distance is used in the knowngeometrical relationship between the pixel distance d, the physicaldistance, and the distance D to determine the value of the ratio R.Using a high percentile is a conservative assumption, meaning that theobject 110 is likely farther away than the distance D, e.g., using a95th percentile means that there is a 95% chance that the object 110 isfarther away than the distance D. This helps keep the brightness of theillumination source 104 within an appropriate range when adjusting thebrightness in a block 445 below.

Remaining with the block 440, for another example, the predefinedrelationship can depend on an orientation of the object 110. If theobject 110 is a human face, the orientation of the object 110 can beoutputted as a result of the facial-detection technique used in thedecision block 410 above. The orientation can be represented as an angleθ of a line between the features of the object 110 corresponding to thelandmarks 114 in the image data 112 with respect to the camera 106. Theangle θ can be used to adjust the pixel distance d when determining thedistance D, e.g., D=R(sin(θ))d, with the ratio R being the same asdescribed above. Alternatively or additionally, the computer 108 can mapthe landmarks 114 to features on a prestored three-dimensional model ofthe object 110 and then use known geometrical relationships to determinethe distance D.

Next, in a block 445, the computer 108 adjusts the brightness of theillumination source 104 based on the distance D. The computer 108adjusts the brightness to a brightness level B. The computer 108 storesa predefined relationship between the distance D and the brightnesslevel B to which the computer 108 adjusts the illumination source 104.For example, the computer 108 can store a lookup table pairing values ofthe distance D with respective values of the brightness level B. Foranother example, the computer 108 can calculate the brightness level Baccording to a stored formula, e.g., B=kD², in which k is a constant.Regardless of how the relationship is stored by the computer 108, thebrightness level B can have an increasing relationship with the distanceD, i.e., the brightness level B gets brighter as the distance D getslonger. As the person is farther away, the illumination source 104 canuse a higher brightness level B while still remaining in an appropriaterange. For example, the brightness level B can increase with the squareof the distance D.

Additionally or alternatively, the brightness level B can be based on anambient brightness B_(amb) in addition to the distance D. For example,the computer 108 can choose the brightness level B to bring a totalbrightness B+B_(amb) to a target level, with the target level increasingwith the square of the distance D, e.g., B=kD²−B_(amb). The ambientbrightness can be determined as described above with respect to thedecision block 415.

Additionally or alternatively, the brightness level B can be based on aspeed V of the vehicle 100 in addition to the distance D. For example,the brightness level B can have an increasing relationship with thespeed V, e.g., B=kD²+f(V), in which f( ) is a function returning apositive value. As the speed V increases, the total exposure of theobject to the illumination source 104 decreases, permitting a greaterbrightness level B.

Additionally or alternatively, the brightness level B can be based on areflectance R of the object in addition to the distance D. Highlyreflective objects can cause oversaturation if the brightness level B istoo high. For example, the brightness level B can be a lower of anappropriate brightness level B_(unc) and a brightness level B_(sat)below which oversaturation does not occur, e.g., B=min(B_(unc),B_(sat)). The appropriate brightness level B_(unc) can be calculated asdescribed in the previous examples for the brightness level B. Thesaturation brightness level B_(sat) can be a function of reflectance Rof the object and the distance D to the object, i.e., B_(sat)=f(R, D).The reflectance R can be a value stored in memory for the type of theobject, e.g., for a face, for another vehicle, etc. The saturationbrightness level can have a positive relationship with the distance Dand an inverse relationship with the reflectance R.

Next, in a block 450, the computer 108 illuminates at least oneadditional illumination source 104 besides the illumination source 104adjusted in the block 445 while receiving new image data 112 from thecamera 106. The additional illumination sources 104 shine in directionsthat encompass the object 110, thus the object 110 is illuminated fromillumination sources 104 at multiple angles. Alternatively oradditionally, the computer 108 may turn off one of the illuminationsources 104 that is currently on while receiving new image data 112 fromthe camera 106.

Next, in a decision block 455, the computer 108 performs objectrecognition on the image data 112 of the object 110, e.g., the imagedata 112 received while multiple illumination sources 104 areilluminated at the object 110 in the block 450. For example, if theobject 110 is a human face, the computer 108 can perform facialrecognition to determine whether the face is a recognized face, i.e., aface stored in memory of a known person such as an owner or operator ofthe vehicle 100. The computer 108 can use any suitablefacial-recognition technique, e.g., template matching; statisticaltechniques such as principal component analysis (PCA), discrete cosinetransform, linear discriminant analysis, locality preservingprojections, Gabor wavelet, independent component analysis, or kernelPCA; neural networks such as neural networks with Gabor filters, neuralnetworks with Markov models, or fuzzy neural networks; etc. If theobject 110, e.g., face, is not a recognized object 110, e.g., recognizedface, then the process 400 ends without actuating any vehicle components118 as described below with respect to a block 465. If the object 110,e.g., face, is a recognized object 110, e.g., recognized face, then theprocess 400 proceeds to a decision block 460.

In the decision block 460, the computer 108 determines whether at leastone feature of the object 110 is three-dimensional. For example, thecomputer 108 can compare shadows from the image data 112 received beforeand after illuminating (or turning off) an additional illuminationsource 104 in the block 450. For example, if the object 110 is a face,then the feature can be the nose, and the computer 108 can compare theshadows cast by the nose. If the shadows have changed, then the object110 is deemed three-dimensional. If the shadows are similar for bothillumination situations, then the recognized object 110 may be aspoofing attempt, e.g., a picture of a recognized face held up to thecamera 106 and not an actual face. If the feature of the object 110 isnot three-dimensional, the process 400 ends without actuating anyvehicle components 118 as described below with respect to the block 465.If the feature of the object 110 is three-dimensional, then the process400 proceeds to the block 465.

In the block 465, the computer 108 actuates at least one of the vehiclecomponents 118. For example, the computer 108 can instruct the doorlocks 120 to unlock. For another example, the computer 108 can adjustone of the seats 122 to a predetermined arrangement. The predeterminedarrangement can be stored in memory paired with the recognized face. Foranother example, the computer 108 can activate the climate-controlsystem 124.

Next, in a decision block 470, the computer 108 determines whether thevehicle 100 is still running. If the vehicle 100 is still running, theprocess 400 returns to the block 405 to continue receiving image data112 from the camera 106. If the vehicle 100 has been turned off, theprocess 400 ends.

In general, the computing systems and/or devices described may employany of a number of computer operating systems, including, but by nomeans limited to, versions and/or varieties of the Ford Sync®application, AppLink/Smart Device Link middleware, the MicrosoftAutomotive® operating system, the Microsoft Windows® operating system,the Unix operating system (e.g., the Solaris® operating systemdistributed by Oracle Corporation of Redwood Shores, Calif.), the AIXUNIX operating system distributed by International Business Machines ofArmonk, N.Y., the Linux operating system, the Mac OSX and iOS operatingsystems distributed by Apple Inc. of Cupertino, Calif., the BlackBerryOS distributed by Blackberry, Ltd. of Waterloo, Canada, and the Androidoperating system developed by Google, Inc. and the Open HandsetAlliance, or the QNX® CAR Platform for Infotainment offered by QNXSoftware Systems. Examples of computing devices include, withoutlimitation, an on-board vehicle computer, a computer workstation, aserver, a desktop, notebook, laptop, or handheld computer, or some othercomputing system and/or device.

Computing devices generally include computer-executable instructions,where the instructions may be executable by one or more computingdevices such as those listed above. Computer executable instructions maybe compiled or interpreted from computer programs created using avariety of programming languages and/or technologies, including, withoutlimitation, and either alone or in combination, Java™, C, C++, Matlab,Simulink, Stateflow, Visual Basic, Java Script, Python, Perl, HTML, etc.Some of these applications may be compiled and executed on a virtualmachine, such as the Java Virtual Machine, the Dalvik virtual machine,or the like. In general, a processor (e.g., a microprocessor) receivesinstructions, e.g., from a memory, a computer readable medium, etc., andexecutes these instructions, thereby performing one or more processes,including one or more of the processes described herein. Suchinstructions and other data may be stored and transmitted using avariety of computer readable media. A file in a computing device isgenerally a collection of data stored on a computer readable medium,such as a storage medium, a random access memory, etc.

A computer-readable medium (also referred to as a processor-readablemedium) includes any non-transitory (e.g., tangible) medium thatparticipates in providing data (e.g., instructions) that may be read bya computer (e.g., by a processor of a computer). Such a medium may takemany forms, including, but not limited to, non-volatile media andvolatile media. Non-volatile media may include, for example, optical ormagnetic disks and other persistent memory. Volatile media may include,for example, dynamic random access memory (DRAM), which typicallyconstitutes a main memory. Such instructions may be transmitted by oneor more transmission media, including coaxial cables, copper wire andfiber optics, including the wires that comprise a system bus coupled toa processor of a ECU. Common forms of computer-readable media include,for example, a floppy disk, a flexible disk, hard disk, magnetic tape,any other magnetic medium, a CD-ROM, DVD, any other optical medium,punch cards, paper tape, any other physical medium with patterns ofholes, a RAM, a PROM, an EPROM, a FLASH-EEPROM, any other memory chip orcartridge, or any other medium from which a computer can read.

Databases, data repositories or other data stores described herein mayinclude various kinds of mechanisms for storing, accessing, andretrieving various kinds of data, including a hierarchical database, aset of files in a file system, an application database in a proprietaryformat, a relational database management system (RDBMS), a nonrelationaldatabase (NoSQL), a graph database (GDB), etc. Each such data store isgenerally included within a computing device employing a computeroperating system such as one of those mentioned above, and are accessedvia a network in any one or more of a variety of manners. A file systemmay be accessible from a computer operating system, and may includefiles stored in various formats. An RDBMS generally employs theStructured Query Language (SQL) in addition to a language for creating,storing, editing, and executing stored procedures, such as the PL/SQLlanguage mentioned above.

In some examples, system elements may be implemented ascomputer-readable instructions (e.g., software) on one or more computingdevices (e.g., servers, personal computers, etc.), stored on computerreadable media associated therewith (e.g., disks, memories, etc.). Acomputer program product may comprise such instructions stored oncomputer readable media for carrying out the functions described herein.

In the drawings, the same reference numbers indicate the same elements.Further, some or all of these elements could be changed. With regard tothe media, processes, systems, methods, heuristics, etc. describedherein, it should be understood that, although the steps of suchprocesses, etc. have been described as occurring according to a certainordered sequence, such processes could be practiced with the describedsteps performed in an order other than the order described herein. Itfurther should be understood that certain steps could be performedsimultaneously, that other steps could be added, or that certain stepsdescribed herein could be omitted.

All terms used in the claims are intended to be given their plain andordinary meanings as understood by those skilled in the art unless anexplicit indication to the contrary in made herein. In particular, useof the singular articles such as “a,” “the,” “said,” etc. should be readto recite one or more of the indicated elements unless a claim recitesan explicit limitation to the contrary. The adjectives “first,”“second,” and “third” are used throughout this document as identifiersand are not intended to signify importance, order, or quantity. Use of“in response to” and “upon determining” indicates a causal relationship,not merely a temporal relationship.

The disclosure has been described in an illustrative manner, and it isto be understood that the terminology which has been used is intended tobe in the nature of words of description rather than of limitation. Manymodifications and variations of the present disclosure are possible inlight of the above teachings, and the disclosure may be practicedotherwise than as specifically described.

The invention claimed is:
 1. A system comprising: an illuminationsource; a camera arranged to detect illumination from the illuminationsource; and a computer communicatively coupled to the illuminationsource and the camera; wherein the computer is programmed to: detect anobject in image data from the camera; identify landmarks of the objectin the image data; determine a distance from the camera to the objectbased on a pixel distance between the landmarks in the image data;adjust a brightness of the illumination source based on the distance;and after adjusting the brightness of the illumination source andreceiving new image data, perform object recognition on the new imagedata of the object to recognize the object as a recognized specificindividual object.
 2. The system of claim 1, wherein the illuminationsource is configured to produce illumination outside a visible range. 3.The system of claim 2, wherein the illumination source is configured toproduce infrared illumination.
 4. A computer comprising a processor anda memory storing instructions executable by the processor to: detect anobject in image data from a camera; identify landmarks of the object inthe image data; determine a distance from the camera to the object basedon a pixel distance between the landmarks in the image data; adjust abrightness of an illumination source based on the distance, theillumination source arranged to produce illumination detectable by thecamera; and after adjusting the brightness of the illumination sourceand receiving new image data, perform object recognition on the newimage data of the object to recognize the object as a recognizedspecific individual object.
 5. The computer of claim 4, wherein theobject is a human face.
 6. The computer of claim 4, wherein theinstructions further include instructions to actuate a vehicle componentupon the object recognition recognizing the object as a recognizedspecific individual object.
 7. The computer of claim 5, wherein theinstructions further include instructions to determine whether a featureof the face is three-dimensional.
 8. The computer of claim 7, whereinthe illumination source is a first illumination source, and theinstructions further include instructions to illuminate a secondillumination source while receiving the new image data.
 9. The computerof claim 4, wherein adjusting the brightness of the illumination sourceincludes adjusting the brightness to a brightness level, and thebrightness level has an increasing relationship with the distance. 10.The computer of claim 4, wherein determining the distance from thecamera to the object includes multiplying the pixel distance by aprestored distance-to-pixel-distance ratio.
 11. The computer of claim10, wherein the prestored distance-to-pixel-distance ratio is based onat least a 90th percentile size of a statistical distribution of objectsof a same type as the object.
 12. The computer of claim 4, wherein theinstructions further include instructions to determine whether theobject is sufficiently illuminated for identifying the landmarks of theobject.
 13. The computer of claim 12, wherein the instructions furtherinclude instructions to increase the brightness of the illuminationsource upon determining that the object is insufficiently illuminatedfor identifying the landmarks of the object.
 14. The computer of claim13, wherein the instructions further include instructions to decreasethe brightness of the illumination source within a prestored durationafter increasing the brightness.
 15. The computer of claim 14, whereinthe prestored duration is at most a duration in which the camera cancapture five frames.
 16. The computer of claim 14, wherein increasingthe brightness includes increasing the brightness to a prestoredbrightness level, and the instructions further include instructions toprevent the brightness of the illumination source from being set to theprestored brightness level for longer than the prestored duration. 17.The computer of claim 12, wherein increasing the brightness includesincreasing the brightness at a prestored rate while receiving additionalimage data.
 18. The computer of claim 17, wherein the instructionsfurther include instructions to maintain the brightness at a currentbrightness level upon determining that the object is sufficientlyilluminated for identifying the landmarks of the object.
 19. A methodcomprising: detecting an object in image data from a camera; identifyinglandmarks of the object in the image data; determining a distance fromthe camera to the object based on a pixel distance between the landmarksin the image data; adjusting a brightness of an illumination sourcebased on the distance, the illumination source arranged to produceillumination detectable by the camera; and after adjusting thebrightness of the illumination source and receiving new image data,performing object recognition on the new image data of the object torecognize the object as a recognized specific individual object.
 20. Thecomputer of claim 4, wherein the instructions further includeinstructions to actuate a component upon the object recognitionrecognizing the object as a recognized specific individual object.